Distribution-Free Learning of Graphical Model Structure in Continuous Domains

In this paper we present a probabilistic non-parametric conditional independence test of X and Y given a third variable Z in domains where X, Y , and Z are continuous. This test can be used for the induction of the structure of a graphical model (such as a Bayesian or Markov network) from experimental data. We also provide an effective method for calculating it from data. We show that our method works well in practice on artificial benchmark data sets constructed from a diverse set of functions. We also demonstrate learning of the structure of a graphical model in a continuous domain from real-world data, to our knowledge for the first time using independence-based methods and without any distributional assumptions. 1.1 Motivation and Related Work Conditional independence of X and Y given a third variable Z is defined as independence of X and Y for every value z of Z almost surely i.e., except a subset of zero probability (Lauritzen, 1996). In this paper we address the problem of testing for conditional independence when the variables X, Y , and Z are continuous. Such a test can be used as a key building block in a wide range of algorithms, ranging from from simple variable selection to learning the structure of a graphical model from data. The latter is our main motivation in this paper. Existing tests depend on strong assumptions on the underlying distribution, such as linear models with Gaussian errors (e.g., Spirtes et al. (1993)). There are many real-life cases where these assumptions fail (e.g., stock market prices, biometric variables, weather status, etc). We do not make such distributional assumptions and thus our test is non-parametric. One class of algorithms for learning the structure of graphical models (such as Bayesian and Markov networks (Pearl, 1997)) from data uses the fact that it implies that a set of conditional independence statements hold in the domain it is modeling. They exploit this property by using the outcome of a set of statistical conditional independence tests to make inferences about the structure. Assuming no errors in these tests, the idea is to constrain, if possible, the set of possible structures that satisfy the conditional independencies that are present in the data to a singleton, and infer that structure as the only possible one.1 For this reason these algorithms are called constraint-based or independence-based. Testing for conditional independence in discrete domains is straightforward. Perhaps the most common method is the χ2 (chi-square) test of independence. In continuous domains, without making distributional assumptions the problem is more difficult. The standard approach is to discretize the continuous variables and perform a discrete test. However, this has to be done with care. For example, Fig. 1.1 depicts two very different situations where X and Y are dependent (left) and independent (right) produce two very similar histograms. The multi-resolution test of Margaritis and Thrun (2001), outlined in the next section, has been developed to address cases such as this. In this paper we extend it to a conditional version that carefully discretizes the Z axis, performs an independence test in each This is provably correct under certain important assumptions. These are: (a) No unobserved variables exist in the domain, (b) variables are conditionally independent of their non-descendants given their parents, and (c) parameters do not take certain zerosupport values that introduce accidental independencies not implied by the structure.